Data structures via the cli supporting docs #1017
Conversation
✅ Deploy Preview for snowplow-docs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
There was a problem hiding this comment.
Looks good overall, but I think we don't communicate anywhere that you need to edit the fiels in place, instead of creating new files for each version. It's kinda implied by names not having the versions, but I think we still want to mention it somewhere. But maybe the recipe will have it
| sidebar_position: 2 | ||
| sidebar_custom_props: | ||
| offerings: | ||
| - bdp |
There was a problem hiding this comment.
Should cloud be included as well?
There was a problem hiding this comment.
as far as i can tell this is just how the docs differentiate between BDP and Community. There are no other options.
There was a problem hiding this comment.
Maybe I'm mixing something up, or the docs were changed since the last time I've done anything there
|
|
||
| 1. Is each file a valid format (yaml/json) with expected fields | ||
| 2. Does the schema in the file conform to [snowplow expectations](/docs/understanding-your-pipeline/schemas/#the-anatomy-of-a-schema) | ||
| 3. Given the organization's [loading configuration](http://localhost:3000/docs/storing-querying/loading-process/) will any schema version number choices have a potentially negative effect on data loading |
There was a problem hiding this comment.
Probably link needs host update
|
|
||
| * A deployed Snowplow BDP pipeline | ||
| * [snowplow-cli](/docs/understanding-tracking-design/managing-your-data-structures/cli/#download) downloaded and configured | ||
| * A familiarity with [git](https://git-scm.com/) and an understanding of [github workflows](https://docs.github.com/en/actions/writing-workflows) |
There was a problem hiding this comment.
I think the product is called github actions, but Im not sure what's the difference
There was a problem hiding this comment.
I thought that too but their docs are a bit confusing. They refer to workflows in a lot of places then actions in their more detailed stuff.
There was a problem hiding this comment.
Don't know, need some marketing people. I would use actions since it's a more recognisable name
There was a problem hiding this comment.
As far as I can tell, actions is the platform and workflows are the things you create. Yeah, I can change it to actions to avoid anyone who hasn't read the definitions getting confused.
|
|
||
| - run: curl -L -o snowplow-cli https://github.com/snowplow-product/snowplow-cli/releases/latest/download/snowplow-cli_linux_x86_64 && chmod u+x snowplow-cli | ||
|
|
||
| - run: ./snowplow-cli ds publish dev --managed-from $GITHUB_REPOSITORY |
There was a problem hiding this comment.
Should we move the --managed-from to be an env variable as well?
There was a problem hiding this comment.
maybe? I don't think I feel strongly. It kind of does fit with things like org id, yeah, id be ok with it either way
There was a problem hiding this comment.
I think it makes sense, it makes the command a bit cleaner
docs/understanding-tracking-design/managing-your-data-structures/cli/index.md
Outdated
Show resolved
Hide resolved
docs/understanding-tracking-design/managing-your-data-structures/cli/index.md
Outdated
Show resolved
Hide resolved
|
@jamessnowplow can we add an explicit example where you request someone else's approval on a new data structure e.g. a data team member? I know it's implied by managing stuff via Github, but it would be really good to call out an approval workflow specifically (e.g. where Contributor A does not have merge approval and Reviewer B does) |
|
@johnmicahreid I'm a bit worried about accidentally writing more about github setup and workflows than snowplow-cli. What level of knowledge are we assuming here? Would the example include how to setup approval workflows and branch rules and prs in github? Or try to capture the essence in a series of screenshots or something? |
work in progress